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ABSTRACT 



The four papers in this volume are four parts of a series on 
the status of recent work on measuring instruction, curriculum content, and 
instructional resources. These papers are intended to identify aspects of 
instruction, to analyze the approaches used in several leading studies, and 
to describe the implications of recent work for National Center for Education 
Statistics (NCES) data collection efforts. The first paper, "Measuring 
Instruction: The Status of Recent Work," by Mary S. Leighton, considers the 
role of instruction in "opportunity to learn" and attempts to assess 
instruction as part of "opportunity to learn." Implications of recent efforts 
to assess instruction for NCES data collection are discussed. The second 
paper, "Measuring Curriculum Content: The Status of Recent Work," by Mary S. 
Leighton and John E. Mullens considers validating national curriculum 
indicators and educational reform in curriculum. "Measuring Instructional 
Resources: The Status of Recent Work" by John E. Mullens, Lisa K. Weiner, 
Angela S. Williams, and Mary S. Leighton considers instructional materials 
and equipment, including computer-related educational technology. The final 
paper, "Measuring Opportunity To Learn: Advancing the State of the Art, " by 
Mary S. Leighton, Brenda J. Turnbull, and John E. Mullens, discusses NCES 
studies of "opportunity to learn, " focusing on elementary and secondary 
education, eighth-grade mathematics, fourth-grade language arts, and U.S. 
History. Each paper contains references. (Contains 13 tables and 2 exhibits.) 
(SLD) 
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Foreword 



Each year a large number of written documents are generated by NCES staff and 
individuals commissioned by NCES which provide preliminary analyses of survey results and 
address technical, methodological, and evaluation issues. Even though they are not formally 
published, these documents reflect a tremendous amount of unique expertise, knowledge, and 
experience. 

The Working Paper Series was created in order to preserve the valuable information 
contained in these documents and to promote the sharing of valuable work experience and 
knowledge. However, these documents were prepared under different formats and did not 
undergo vigorous NCES publication review and editing prior to their inclusion in the series. 
Consequently, we encourage users of the series to consult the individual authors for citations. 

To receive information about submitting manuscripts or obtaining copies of the series, 
please contact Suellen Mauchamer at (202) 219-1828 or U.S. Department of Education, Office 
of Educational Research and Improvement, National Center for Education Statistics, 555 New 
Jersey Ave., N.W., Room 400, Washington, D.C. 20208-5652. 



Susan Ahmed 

Acting Associate Commissioner 
Statistical Standards and 

Methodology Division 



Samuel S. Peng 
Statistical Service and 

Methodological Research 
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Preface 



The four papers contained in this volume are four parts of series on the status of 
recent work on measuring instruction, curriculum content, and instructional resources. These 
papers are intended to identify aspects of instruction, to analyze the approaches used by 
several leading studies, and to describe the implications of recent work for NCES data 
collection efforts related to the opportunity to learn. The work documented in these reports 
was conducted by Policy Studies Associates, Inc. under Contract No. RN 9306100. 

The authors of the papers presented in this volume are Mary S. Leighton and John E. 
Mullens, Brenda Turnbull, Lisa K. Weiner, and Angela S. Williams of Policy Studies 
Associates, Inc. 
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MEASURING INSTRUCTION: 
THE STATUS OF RECENT WORK 



This paper has three purposes: 

• To identify and list the aspects of instruction that shape students’ opportunities 
to learn 

• To analyze the approaches used by several leading studies to assess instruction 
in order to characterize opportunities to learn 

• To describe the implications of recent work for NCES data collection efforts 
related to opportunity to learn. 



The first section of the paper summarizes research findings about features of instruction that 
contribute to learning. It begins with process/product research results and then explains alternate 
approaches that add validity to the insights gained from correlational and experimental studies. The 
section concludes with an excerpt from a description of a second grade math class and a discussion of 
how analytic frames capture different aspects of the event. The second section of the paper describes 
how contemporary studies have approached assessment of opportunity to learn. It categorizes study 
questions and data collection strategies, and reports on the effectiveness of different strategies (to the 
extent that information is available). It shows how the information produced by these strategies maps 
onto the framework described in the first section, to demonstrate the extent to which they capture 



what other studies suggest are the determining aspects of instruction. The final section of the paper 
explores how the frameworks for understanding the contribution of instruction to opportunity to learn 
and recent experiences in assessing instruction can inform NCES efforts to assess opportunity to learn 
as a part of its national data collection programs. 



The Role of Instruction in Opportunity to Learn 

For the purposes of this series of papers, we have characterized opportunity to learn as a 
construct with three components: curriculum content, instruction, and instructional resources (such as 
materials, computers, texts, and labs). Certainly what students learn in any course depends in pan on 
what was covered; if a topic never arose in school, student mastery could not be assumed as a 
consequence of schooling. The quantity and quality of instruction also influences learning. If a topic 



is covered too hastily or with erroneous explanations, for instance, it may leave no more trace in 
student achievement than if it were not covered at ail. Likewise, the availability of resources to 
support learning-such as library books, online information retrieval systems, desks, and supplies- 
influences productivity. 

These components have overlapping elements; in practice, aspects of one may also be aspects 
of another. "Curriculum content” includes not only major and minor topics covered, but also 
teachers’ emphases and role orientation and student performance expectations, because, although they 
may be considered instructional rather than curricular variables, the latter features create operational 
definitions of each topic. For instance, a lesson may cover the concept of least common denominator, 
but how students experience the me anin g of that concept will be influenced by whether it is a small 
digression in a bigger discussion of subtracting fractions, whether the teacher acts according to a 
conviction that her role is to stimulate discovery (rather than to explain), and whether as a result of 
the lesson students are expected to solve problems or only to name the common factors of two 
numbers. All of these cues communicate what the lesson is about, which is likely what the students 
will engage in learning. The curriculum content papers also touched on resource availability, to the 
extent of reporting the use of supplies, equipment, and facilities as instances of lesson content. In this 
paper, we examine the dimensions of instruction that sound research shows to have a clear bearing on 
student achievement and thus should be construed as influencing opportunities to learn. 



The Results of Process/Product Research 1 

Process/product research examines the relationship between certain of teachers’ overt and 
quantifiable instructional behaviors and the achievement of typical students. Extensive correlational 
studies, often supported by experimental studies and replicated in many settings, have identified four 
areas of instruction that significantly influence students’ opportunities to learn across grade levels and 
subject matter: 



Amount and pace of instruction 
Presentation of information 
Questioning strategies 



1 The material in this section comes mainly from Brophv and Good’s (1986) comprehensive 
research summary, which appeared in the Third Handbook of Research on Teaching . 



Reactions to student responses 






Some evidence also suggests that particular features of seacwork and homework assignments make 
small but significant contributions to learning. The constitution of the instructional group— whether 
whole class, small group, or individual-has also been studied for its relationship to learning. 

Until recently, many comprehensive studies of instructional effectiveness focused on models 
of direct teaching, and their findings apply most neatly to traditional teacher-centered, whole-class 
lessons. However, some findings have implications for other models of teaching that may involve 
more student-teacher or student-student interaction. In the following sections, such extrapolations 
have been suggested whenever they seemed supportable. 

Amount and pace of instruction. According to Brophy and Good, "The most consistently 
replicated findings link achievement to the quantity and pacing of instruction" (1986, p. 360). 
Conventional measures such as the number of text pages covered or test items taught show strong, 
positive correlations with student achievement. Teachers who give top priority to academic activities 
and who allocate the most time to academics nurture the most learning among students. Maintaining 
an environment that supports students’ sus tain ed engagement in learning also contributes to 
achievement. The rewards of sustained engagement depend in part on students’ success with the 
assigned tasks-if work is too easy, given the circumstances, it occupies students without advancing 
their knowledge or skill. Alternatively, if it is too difficult, it may hold their attention without 
generating understanding. The level of difficulty appropriate for different conditions varies. Very 
hard work may challenge and motivate a group of students who are deeply interested in a topic, while 
periodic assignments of very easy work-perhaps after a stretch of challenging exercises— may 
consolidate learning and promote facility. In general, active teaching strategies— those that (1) include 
stimulating exchanges between the teacher and students and among the students. (2) involve the 
teacher deeply as a mediator of learning, and (3) ensure that students experience high-quality 
representations of content— promote more learning. (This is in contrast to relatively unproductive 
situations in which students work unassisted trying to make sense of textbook explanations or other 
content exposure without the teacher’s oversight to catch misconceptions or misinterpretations and 
monitor comprehension.) 

Presentation of information. In lessons where the teacher serves as the primary source of 
information, usually by lecturing, demonstrating, and eliciting student recitation, several features 
boost student learning. Structuring material with advance organizers, periodic summaries of what has 
been done and what is ahead, and explicit reference to patterns as they emerge facilitate retention and 
give students better access to the big picture. Planned redundancy and careful sequencing of lesson 
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